Repository navigation
[Klaud Cold] minimaxm31flash-fp4-h200-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash H200 DSpark recipe - #4423
Conversation
…lash AgentX on H200 Hopper TP8 recipe from recipes.vllm.ai: FP8 KV and indexer caches, NVFP4 experts on Marlin MoE, block size 128, FULL_AND_PIECEWISE graphs, plus the GB200 recipe's three-token DSpark. Context kept at 1M for the uncapped AgentX corpus. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the contribution!
中文感谢你的贡献!
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| minimaxm31flash-fp4-h200-vllm-agentic-dspark: | ||
| image: vllm/vllm-openai:minimax-m31-flash@sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c | ||
| model: MiniMaxAI/MiniMax-M3.1-Flash | ||
| model-prefix: minimaxm31flash | ||
| runner: cluster:h200-dgxc | ||
| precision: fp4 | ||
| framework: vllm | ||
| multinode: false | ||
| srt-recipe-dir: minimaxm31flash/vllm/h200-fp4-mtp | ||
| scenarios: | ||
| agentic-coding: | ||
| - dram-utilization: 0.80 | ||
| search-space: | ||
| # TP8 is the upstream Hopper layout. The 250 GB checkpoint leaves ~88 GiB of | ||
| # KV per 141 GB GPU, about 1.3x MiniMax-M3 H200 (run to c14). | ||
| - { tp: 8, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 12, 16, 20, 24], srt-recipe: agentic.yaml } |
There was a problem hiding this comment.
🔴 Every throughput run of this new config will fail before benchmarking, because its golden acceptance curve is not in the repo. The vLLM speculative-config is dspark, so the srt driver looks up minimaxm31flash_dspark.yaml in infx/golden_al_distribution/. That lookup raises "No committed golden curve" (curves.py:68), and the PR description confirms the file is not committed. Fix: commit minimaxm31flash_dspark.yaml in this PR, or hold back the master entry and its perf-changelog entry until the curve lands, so no run can be scheduled without it.
Why this was flagged
Any non-eval agentic run of minimaxm31flash-fp4-h200-vllm-agentic-dspark has SPEC_DECODING=mtp (nvidia-master.yaml:5295). It also has a vLLM speculative-config with method dspark (agentic.yaml:44). So acceptance_length in infx/srt_slurm/synthetic_acceptance.py calls golden_length("minimaxm31flash", spec, ...). load_curve raises ValueError "No committed golden curve minimaxm31flash_dspark.yaml in ..." at infx/golden_al_distribution/curves.py:68. The only files under golden_al_distribution/ for this family are minimaxm3_eagle3.yaml and minimaxm3_eagle3_gqa.yaml. Nothing checks this before merge: only the golden_al_distribution unit tests use GOLDEN_DIR, and they do not cross-check master configs. The perf-changelog entry adds this key to the benchmark set, so once merged every scheduled sweep point (8 concurrencies) fails at override construction and produces no results. MODELS.md and the changelog also cite a curve that does not exist. The PR description says the sweep is "on hold", but nothing in the code enforces that hold.
Verification: Trigger: any non-eval agentic throughput run of this config key. There is no minimaxm31flash_dspark.yaml in infx/golden_al_distribution/. load_curve raises at curves.py:68, and main returns 1 before any srtctl apply is submitted. No test checks master entries against the curve files, so CI will not catch this before merge.
| - "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + MXFP8 dense) AgentX on H200 with vLLM image minimax-m31-flash (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)." | ||
| - "Serve flags follow the recipes.vllm.ai Hopper TP8 recipe: block size 128, FP8 KV and indexer cache, Marlin MoE backend for the NVFP4 experts, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, three-token DSpark (FLASH_ATTN drafter, fp8 draft KV, block rejection, from the GB200 recipe; upstream does not enable DSpark on Hopper). Context stays at 1M (upstream Hopper example uses 256K) because the AgentX corpus for this prefix is uncapped." | ||
| - "Search space: TP8 only, conc 1-24, no KV offloading. Throughput runs use synthetic acceptance from the minimaxm31flash_dspark golden curve." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The changelog validator rejects this new entry, so the PR's changelog check fails. pr-link: TBD is neither the canonical PR URL nor an accepted placeholder. Fix: use pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or XXX, or the real PR URL. Those are the only values validate_added_pr_link accepts.
Why this was flagged
validate_added_pr_link in inferencex-e2e/infx/workflows/validate_perf_changelog.py:139-150 checks new PR entries. On PR runs it allows only values in PR_LINK_PLACEHOLDERS ("XXX" and "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX", lines 22-25) or the expected PR URL. Anything else raises "new PR entry must use ... or an XXX placeholder; found 'TBD'". On main, where pr_number is None, it requires CANONICAL_PR_LINK, so TBD fails there too. The appended block at perf-changelog.yaml:10881 sets pr-link: TBD, so changelog validation fails for this PR. The merge-time canonicalisation in merge_with_reuse also rewrites only recognised placeholders, so it will not fix this value.
Verification: The appended block ends with pr-link: TBD (perf-changelog.yaml diff, new entry). ChangelogEntry.pr_link is a plain str (infx/matrix/validation.py:1062), so the PR's own changelog check does not reject TBD. The rejection actually happens at merge. Result: the supported merge flow refuses this PR until pr-link is changed to XXX or the real URL.
| served-model-name: MiniMaxAI/MiniMax-M3.1-Flash | ||
| language-model-only: true | ||
| trust-remote-code: true | ||
| reasoning-parser: minimax_m3 | ||
| tool-call-parser: minimax_m3 | ||
| enable-auto-tool-choice: true |
There was a problem hiding this comment.
🟡 (optional) Throughput results may pair a thinking_on golden acceptance length with replay traffic served in the template's default thinking mode. That mode may not be thinking-on. Every MiniMax-M3 vLLM AgentX recipe pins default-chat-template-kwargs: '{"thinking_mode":"enabled"}', but this new recipe's args do not. The AgentX client sends no thinking toggle. Fix: add default-chat-template-kwargs: '{"thinking_mode":"enabled"}' to roles.agg.args, or confirm the M3.1-Flash template defaults to thinking-on. Either way the served mode matches the thinking_on curve the driver picks.
Why this was flagged
The condition is the MiniMax-M3.1-Flash chat template not defaulting to thinking-on. I could not check this from the checkout, and the M3 template's toggle is the thinking_mode kwarg (speedbench/minimaxm3_fp4_b300_vllm.sh:43-44). The srt planner hard-codes THINKING_MODE = "thinking_on" (infx/srt_slurm/generate.py:32). That value selects the golden curve in acceptance_length (infx/srt_slurm/synthetic_acceptance.py:131). The recipe args at agentic.yaml:30-58 set no default-chat-template-kwargs. A grep of infx/bench finds no chat_template_kwargs or thinking field in the AgentX client. So the server falls back to the template's default mode. All seven MiniMax-M3 vLLM/ATOM agentic recipes (for example minimaxm3/vllm/h200-fp8-mtp/agentic.yaml:40) pin thinking_mode enabled. perf-changelog.yaml:6361 explains the pin as "Pin thinking-on so the run matches the golden ... thinking_on" acceptance target. If the default is off, the new config publishes numbers whose output-length profile does not match the thinking_on AL it simulates. No validator compares the two.
Verification: Trigger: the MiniMax-M3.1-Flash chat template does not default to thinking-on. The new recipe's roles.agg.args (agentic.yaml lines 30-58) set no default-chat-template-kwargs. infx/srt_slurm/generate.py:32 hard-codes THINKING_MODE = "thinking_on". In that case the throughput numbers model thinking-on acceptance on traffic that is not in thinking mode.
| # MiniMax-M3.1-Flash AgentX on H200: day-zero vLLM image from | ||
| # https://vllm.ai/blog/2026-10-23-minimax-m31-flash, multiarch manifest verified | ||
| # 2026-10-23 (amd64 is minimax-m31-flash-x86_64-cu134). Serve flags follow the | ||
| # Hopper TP8 recipe: FP8 KV and indexer caches, NVFP4 experts on Marlin MoE. | ||
| # Upstream does not enable DSpark on Hopper; this adds the GB200 recipe's | ||
| # three-token DSpark, with synthetic acceptance from minimaxm31flash_dspark.yaml. |
There was a problem hiding this comment.
🟡 (optional) These new comments break the mandatory AGENTS.md comment rules, which forbid header comments and history or provenance comments. The block above the master entry describes the config and records where it came from ("day-zero vLLM image from ", "verified 2026-10-23"). Fix: delete the header and provenance text, keep only short per-line technical reasons, and leave the history to perf-changelog, which covers the 4 sites listed. Same pattern at 4 sites (configs/nvidia-master.yaml:5274-5279, configs/nvidia-master.yaml:5293-5294, h200-fp4-mtp/agentic.yaml:26-29, h200-fp4-mtp/agentic.yaml:65-67).
Why this was flagged
AGENTS.md:64 says never add broad comments above a recipe, override or section that say what the config is. AGENTS.md:65 forbids comments about where something was derived from or about past results. nvidia-master.yaml:5274-5279 is a header above the new entry that cites the blog URL, a verification date and upstream provenance ("Upstream does not enable DSpark on Hopper; this adds the GB200 recipe's..."). nvidia-master.yaml:5293-5294 compares against a past result ("about 1.3x MiniMax-M3 H200 (run to c14)"). agentic.yaml:26-29 says "Serve flags follow the Hopper TP8 recipe at https://recipes.vllm.ai/..." and "DSpark follows the GB200 recipe", which is derivation. agentic.yaml:65-67 is a section comment above the zip_override_tp8 block. The base branch has none of these comments. Nothing enforces the rule automatically, so the comments merge as written.
Verification: nit. Nothing breaks at runtime. The comments break the mandatory comment rules in AGENTS.md. Line 64 says "No header comments... above a recipe, override, or section." Line 65 says "No history... where it was derived from, what it replaced, or past results." The header at nvidia-master.yaml:5274-5279 sits directly above minimaxm31flash-fp4-h200-vllm-agentic-dspark:.
…to match the golden curve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39903574233 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39903574233 |
|
/use 39903574233 |
1 similar comment
|
/use 39903574233 |
|
@Oseltamivir staged run 39903574233. This run remains available across future |
…p-sweep] Keep main's H100 entries and place the H200 master-config block after them; append the H200 changelog entry at the end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Day-zero MiniMax-M3.1-Flash DSpark (
MiniMaxAI/MiniMax-M3.1-Flash) AgentX on H200 with vLLM, from https://vllm.ai/blog/2026-10-23-minimax-m31-flash. This follows the Hopper section of https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash.vllm/vllm-openai:minimax-m31-flash, pinned to the multiarch digestsha256:86f582da…(amd64 =minimax-m31-flash-x86_64-cu134).--block-size 128,--kv-cache-dtype fp8,indexer_kv_dtype: fp8,--moe-backend marlin(NVFP4 experts; Hopper has no FP4 tensor cores),FULL_AND_PIECEWISEgraphs,--disable-hybrid-kv-cache-manager, theminimax_m3parsers, andsafetensors-load-strategy: prefetchfor the h200-dgxc VIRTIOFS cache.FLASH_ATTNdrafter, fp8 draft KV, block rejection). Upstream doesn't enable DSpark on Hopper, so whether the drafter runs on Hopper is still unverified. Throughput runs use synthetic acceptance fromminimaxm31flash_dspark.yaml, which isn't committed yet (an AL curve run is in progress). The sweep is on hold until it lands.max-model-lenis 1M rather than the 256K in the upstream Hopper example, because the AgentX corpus for this prefix is uncapped.h200-dgxc), no KV offload.Siblings: #4422 (H100), #4419 (GB200/GB300), #4420 (MI355X), #4421 (B200/B300).
🤖 Generated with Claude Code